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This paper, which is part of the work of the Stanford 
University Environment for Teaching Program, disdusses problems that 
are repeatedly found in the literature on formative evaluation, with 
emphasis on the special difficulties of conducting formativie 
evaluation in schools, it defines the requirements of effective 
formative evaluation as (1) a link between evaluation aid decision 
making (provided by continuous assessment and f ^dback) , (2) 
involvement of all who will be affected by decisions based on, 
evaluation, (3) a theoretical framework to support the evaluation, 
and (1) close attention to the school's social and political setting. 
On the basis of ' this discussion the papier suggests a survey- feedback 
approach which provides objective information and a process involving 
all relevant constituencies in a dialogue with natural work groups in 
the school to define school problems and develop strategies for their 
solution. This formative evaluation strategy is aimed at 
institutionalizing problem solving in a school, and in this regard 
the paper examines the problems and prospects of the approach. 
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Introductory Stateiqent - 

' % ^ 

^ The Center's mission is to improve teaching in American schools. 
Its work is carried out through three research and development programs- 
Teaching Effectiveness, , the Environment for Teaching, and Teaching and 
Linguistic Pluralism— -and a technical assistance program, the Stanford 
Urban/Rural Uadership Traininc Instituted A program of Exploratory 
and Related Stydi^s includcjs afealler studies not included in the major 
prograjns. The ERIC Clearin^Jfiouse on Information Resources is also a 
part of the Center. 

• . - . 

This paper, which is part of the work of the Environment for 
Teaching Program, discusses problems that' are repeatedly found in the 
literature an formative evaluation, with emphasis on the special dif- 
ficulties of conductiltg fomative evaluation in schools. It defines the 
requirements' of effective f^rm^tive evaluation as (a) a link between 
evaluation And decision making (provided by continuous assessment and 
feedback), (b) involvalnent of all who will be affected by decisions 
based on evaluation, (c) a theoretical framework to support the Evalua- 
tion, and (d) close attention to the school^' s social and political 
setting. On- tl^e basis of this discussion the paper suggests a survey- 
feedback j^pproach whicif" provides objective information and a process 
involvingvall relevant constituencies In a dialogue with natural work 
groups In the school to define school problems and, develop strategies- 
for their Solution. This, formative evaluation strategy is aimed at 
institutionalizing problem solving in a. school, atid in this/ regard the 
•paper examines the problems and prospects of. the' approach. 
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', WATER and' THE DUCK'S BACK: THE USE OF FORMATIVE 
EVALUATION IN SCHOOLS 

Terrence E, Deal and Kathleeri M. Hugueniri >' 

Evaliiktion efforts in schools have had an effect* equivalent to that 
made by water on the bacKof a Suck. There are those ,who would attribute 
responsibility for this situation to ; the duck:*^ "Schools have character- 
istics which make them resistant to Evaluation efforts." And there are 
tKosB who would attribute it to the water: "Today's educator may rely 
little on formal evaluation because its answers have seldom been answers 
to questions he iLs a^sking" (Stake, 1967), 

This. paper assigns the responsibility neither to the wat - nor to 
the duck but to the match between them. It asks the question: riow can 
formative evaluation assist educators in improving' schools? Our purpose 
is to suggest v^some general criteria for forma*tive evaluation and a 
specific strategy for improv^g it. First, we shall define formative \ 
evaluation arid focus on the limitations and problems that are Recur- 
rently described in the literature.^ Second, we shall consider the unique 
.characteristics of school organizations that pose challenges to formatiVe 
evaluation. Third, ^we shall suggest a survey-feedback process as a 
specific strategy for evaluating schools as organizations. Fourth, 
we shall discuss the potential of the sufvey-f eedb^ick approach for 
addressing the problems of formative evaluation. 



Terrence E. Deal is la Research and Developmen|: Associate and a 
Lecturer in the School of\ Education at Stanford. Kathleen M. Huguenin 
is a Research Assiteiii^ht at the Center. - 

A. slightly different version of this paper was presented at the 
Annual Meeting of the Pacific Socio.^ogical Association, Victoria, 
British Columbia, April, 1975. 



Formative Evaluation: \^at Is It? 

In its broad sense, "evaluation is the process of ascertaining the 
decision areaar of concern, selecting appropriate information, and col- ^ 
lecting and analyzing information in ord6r to report summary data use- 
ful to decision makers in selecting among alternatives" (Alkin, 1969). 

Within this broad area of evaluation several finer distinctions 
can be made.^^ The first crucial distinction is between evaluation that 
is conducted while an activity or program is going on and evaluation 
that is conducted after the fact. The former type of . evaluation ;Ls 
formative, the later summative (sometimes called post hoc pr pass-fail]! . 
According to Scriyen, "formative evaluation produces information that ife 
fed back during the development (of a program) to help improve it. 
served the needs of the developers" (In Weiss, 1972). This pape^ focuses 
on formative rather than summative evaluation. - . \ . - 

Another important distinctign is between the goals and roles' of 
evaluation. Recognizing the misunderstanding and confusi6n between the. 
two, Scriven (1967) makes the following dichotomy: goals are the method- 
ological activities that are used Iji the evaluation process,' siflCHOTs^ 
whether they are applied to the- evaluation of coffee machines or teaching 
machines; the roles of evaluation focus on the c<jfitext in whichfthe 
evaluation process is used, or the functioA it plays in a partlculaT 
sit'aation. Implicit in the role of evaluation is a direct ll:nk to^ the ^ 
poll^cy and decision-making process. Objections at the toie^leve^j^f ten 
means, the goal questions do not get answered. Thus this paper is -de- 
signed to focus on an explicit interrelationship between roles^and goals. 

A third distinction is between evaluation research and policy 
analysis. Rossi (1972) defines research as the use of scientific 
methods to describe phenomena and their relationships, wliile policy 
analysis synthesizes- this information and develops guides for decision 
making. Ideally, these two aspects of evaluation would not be mutually 
exclusive but rather complementary. ^"The circumstances under which to 
conduct policy analysis would be when^'the effecti^ess of alternative 
social policies are known through evaluation research" (Rossi, 1972). 



Evaluation incorporates . a broad range of activities. Ideally, 
evaluation research provides'-eithef formative or summ^itive input for^ 
decision makers. It encourages a consideration of the -contributions of 
both the goal of an evaluation and the roles that it can play. And it 
provides a means through which research can contribute to policy analysis. 
Why, then, are so many evaluation stuijies shelved on far corners to 
collect dust? Why does the cycle cease at the analytical level tather 
than lead to recommendations for action? And why the general dis.truQ.t 
and misgivings al^ut evaluation? " - 

Recurrent Problems ' 

— ^ 

Our main concern is why formative evaluation, in particular, has 
not provided a more positive contribution or an imp^us for action in • 
improving schools. We will begin by focusing on its limitations and 
the problems encountered in conducting it. These problems will be 
discussed under four general headings that describe the' major require- 
ments, or needs, of formative evaluation: (1) a linkage between' 
evaluation and decision making, (2) involvement of those who will be 
affected by decisions based on evaluation, (3) the existence of a 
theoretical framework, and (4) attention to the social and political 
setting. 

. Need for continuous assessment and feedback into the decision- ^ , 
making process . This need arises from the following three problems. 
First, the linkage with the dedision-making process is frequently ^ 
absent. By definition, "an evaluation study should be a problem-solving ^ ' 
enterprise wi^ih a clear-cut relationship to some decision-making 

function" (Suchman, 1970). This assertfion is found throughout the 

. - , ^ CP- 

evaluation literature. The primary criteria by which an evaluation's 
effectiveness can be measured are its usefulness and its potential to 
lead to change. Nevertheless, as Cuba (1969) reiterates in his list.v 
of specific problems In. this area, there has been repeated failure to* 
link evaluation and the. decis|.on-making process. 



-4- 



Second, there has been a lack of emphasis on feedback.^ Formative 
evaluation Involves more than a final judgment; It "emphasizes an under- 
'standing' of the ongoing proce^ss and a continual feedback of Information. 
Rather than*a "before" and "after," a "during, during > during" design Is , 
necessary. We maintain that an interactive process — stop, go back, re- 
vise or continue — Is needed to make evaluation useful. Weiss (i972) 
proposes the following remedies, which are designed to Increase the use-" 
fulness of evaluation feedback: it should be given (a) to the appro- ' 
prlate level of us^rs, (b) in^uAderstandable, nontechnical language^ 
(c) ^S^ore decisions are made. ) 

** Third, evaluations are of ten- one-shot and af tei>-the-fact (Cuba, 
1969; Suchman, 1970; and Weiss, 1970). Furthermore, the decision that . 
follows is often dichdtomous: Do we or don't we continue this program? 
' It is repeatedly suggested in the literature that tl^ere be stress on ^ * 
program improvement rather than simply a judgment of success or failure. 

Given these limitations, formative evaluation rarely leads to action, 
and evaluationj results are seldom used. There is no direct link between / 
ass^sment and improvement. If improvement is the main goal of formatl^re 
evaluation then making such a linkage is crucial. ' ^ " 

Need to involve relevant grpups . IVo issues reciir in litera- 
ture: the lack of Involvement of the relevant individuals and groups 
in the olrganizatiorCor^rogram being evaluated, and the concomitant 

reliance on experts from outside. Not only are representatives^ f rom the 
. " ' ' < * '.^ ' 

various levels of a program frequently excluded, but often even admlnis- 

trators-do not take an active role in the evaluation. process . Involve- 
ment of these relevant Individuals and groups is frequently suggested 
as a way of minimizing friction between the evctluators and the staff of 
the program or organization being javaluated and as a potential way of 
Increasing the implementation of action^" suggested by the evaluative 
results. In addition, input from all relevant sources is often suggested 
to* Increase the validity of evaluation findings (Weiss, 1972). These 
possible advantages seldom become reality because the participants are 

usually^,marglnally andl^assively Involved in the evaluation. 

^ • - ■ • 
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Heavy reliance on externaHy tjalned personnel 'creates a gap between 
demanj.and supply, b^ecause adequately trained evaluation personnel are .a 
rarity (Cuba, 1969). ht also reduces the legitimacy of the evaluation, 
.since "-outsiders rarely generate a sense of legitimacy and trust. Hence, 
the involvement of practitioners in the evaluation processlSakes sens^ 
on two levels. If internal personnel are involved they should perceive 
evaluation as less threat^hlpg.- Further, their involvement produces a 
larger volume..of inside information such as the "fcackstage" realities of 
the program,Lthe' norms, the jargon, and the critical subterranean issues 
(Weiss, 1972). 

Need to make the theoretical framework explicit . This need arises 
from four concerns expressed, in the literature. First, evaluation must 
have aii underlying conceptual framework. A particidar atuidy should be 
located in an overall theoretical perspective (Weiss, 197,2). A concep- 
tual framework, as opposed to the individual idi*«xnc'^^s^^s vhlms 
of the evaluator, should determine where the attention i* focused, i.e., 
what information is obtained. It also influences the evaluation method- 
ology— how' information is obtained, and how if is summarized or processed. 
Finally, the conceptual framework has implications for the selection of '* 
an intervention strategy to correct areas identified by the evaluation as 
needing improvement. • There must be a match between the conc^tuall frame- 
Work and the evaluation prpblem. For'^example, where fundamental p^blems 
may exist* in the organizational -structure, evaluation based on a psycho- 
logical model may not focus attention on the most salient aspects '' 

(De\L, 1974). . . 

Second, theire is a .need to make the evaluator 's underlying frame- 
work explicit. Rein (1973) points o«t that our social science per- 
spectives determine how we view the world and how we organize knowledge 
concerningNrhe variables we _ examine. An evaluation model is basically 
a set of assumptions about how a particular segment of the world works. 
C«(nsequently, any evaluation ^tudy needs to state exactly what these 
Assumptions are. Alkin (1969) charges that when an evaluator reports 
summary data to a decision maker, judgments are i-roplicit, and it is 
therefore incumbent upon that evaluator to make eScplicit the value 
systems underlying these jtidgments. 



Third, .the formulation of goals has ^becen overemphasized. Aside 
from the obvious -problems encountered when attemt)tlng to operatlpnallze 
and measure goals, tocx much emphasis on this level ^l^ds to a^ preoccupa- 
tlpn with valuB judgments and Intentions and DK)ves 'away from an examina- 
tion of what 16 actually happening in a particular setting (Scriven, 1967) 



To avQld bei^ bogged down in ^e goal "swamp,", a systems apprpa^lj^ is 
often used* 

• The systems approach, suggested by Etzioni (Weiss, 1970) ' addresses 
a fourth theoretical problem — the limitation of reliance on^ a static, 
rather than a. dynamic, framework. Ba^ic to the system^ approach is an 
image of organizatit)ns ,as dynamic interrelationships among various sub- 
systems. This is congruent, with Suchman's (1^70) assertion that it is 
necessary for evaluators to -view ^r gani za ^tflon s as relationships rather 
than static entities. The systems view has implications for where 
evaluators focus attention. Programriidtic effect^s cannot only be observed 
on participants, but on the .entire system and the vat io us subsystems. 
Implications can be derived for possible organizational resistance to 
changes which the evaluation ultimately suggests. Though logical or v 
rational, these proposed changes may not be organizationally feasible \ 
or acceptable given a holistic view pf organizational interrelationships. 

Need to consider the social and political context . By definition, . 
because evaluation is description and judgment, it is political. Cohen , 
(1970) asserts that evaluation is necessarily political because it often 
contributes to changes . in power relationships.. It produces information 
that is pot.entially relevant to the decision makers' who are responsible 
for allocating resources — moneyl position, or, authority. In the past, 
evaluators have failed to anticipate the ramifications of suggested 

change, strategies and resistance to them. ' 
^ ■ ' • * 

Considerable resistance originates in the social structure of the 

organization and environment in which the evaluation takes place. Within 

any such context there exist potential support and potentid^ obstructions; 

both must be taken into apcount. These socJLal. realities have an effect 

on potential organizational resistance to change.' Longwood and Simmel 

(1962) Kave* noted that "no ma1?ter what purpose an organization is 
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created for, once it is established its purpose becomes to perpetuate 
Itself." Social and political consit^eratioti^an be useful in determin- 
ing whether resistance will /come from li^diViduals within the 'organization, 
^•from th^ sociopolitical-environment in whicH\ the organization is embedded, 
or from the organization it self v 

Some^ Special Problems of Formative Evaluation iH^Sc^ols 

The four geri^ .1 problems of formative., evaluation transfer as 
evaluation is conducted in educational?* settings . Additionally, schools 
Have special features which add to or Intensify these probl6tns. 

Education—teachltiR and learning — takes place in forma l, complex 
oWanizations , It is therefore important that the systems framework 
iderlying the evaluation be closely related to organization theory. 
)rganizatipns can be .considered systems ^th five main subsystems: 
^environment, technology, formal structure, individual and ^11 group 
norms and processes, and goals (Udy, 1965). Systems theory etohasizes 
that these subsystems are interrelated; hence, changes in any cn^ sub-^ 
system have consequences for all the rest. Environmental changes 
example, affect the other organizational subsystems: goals, struc^e, 
technology , and. small group processes . . ^ 

Formative evaluation may- focus on any particular prganizational 
subsystep. But because these subsystems are so highly interrelated in 
schools, as In other (organizations, feedback that suggests changes in ^ 
only one subsystem- may produce problems because the impact on the others 
.was not asse&sed or^because the other subsystems did not provide the 
proper support for the chanjge. For example, an evaluation of a particular 
reading program (or technology) may reveal the need for a higher level of 
Indiyidtiallzatlon. But changiiig thepdadfilg^^^^ may also require 

changes In \t^ way teachers work tog^^^r in the way school-wide de- 
cisions' are. inkde.* : Similarly, assessment of community needs may lead " 
to chiange^ in a st:hool's .educational goals. ^ But these changes may also 
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affect structural features af the school such-a§ evaluation processes, 
specializatilpn, or coordination-needs, and may Increase the demands on 
Indlvlduals^-M-^'-^tiange processes that opej^te )Ln small groups (Deal 
and Ba fridge, 1974). 

The sim of conducting formative evaluation is to. improve schools. 
Since schools are complex organizatior^, sensitivity to the dynamic 
relationships among the five important* subsystems is crucial if improve- 
nvents are to be sustained. The conceptual foundation of formative evalu- 
ation ^ould facilitate sensitivity 6y providing a way of understanding 
the organizational forces that support o-r constrijln both evaluation 
and improvement. 

In addition to the dynamic rfelatdonships ot the main organizationa; 
subsystems, there are some special qualities of each subsystem iu school 
organizations that affect the formative evaluation process. The most 
important implications of these qualities for. evaluation stem from the 
relationship between schools ani their environments and the nature of 
educational goals.' In. any form^ive evaluation the social and pplltical 
context is an important consideration. Schools, however, are.^ t maailly 
controlled by the communities in which they function and by tl^^tate 
legislature . Ultimately, what is taught in schools— ^s well as how it is 
taught an<£^by whom: — is determined by lo,cal school boatds and thd state.. 
The social and political environment is oy^rtly patt of the school 
system. As a result, formative evaluation must pay special attention 

. C « 

to the social configurations and political forces that operate in the 
local community or other levels of the environment. There must also ^e 
a recognition that*^ thfese configurations and forces will vary in diversity, 
stability, and influence. ^ 

The goal subsys^m in schools is also different from other organiza- 
tions in which formative evaluation takes place. The goals of education ' 
are diffuse and multi- faceted . In .a business organization the ultimate 
'goal is profit, which is specific, unidimensionaj., and easy to measure. 
In comparison th^e goals of education are unclear and diverse, and 

measuring progress toward them is a, feat that outdistances current 

■ - ' ■ .... 

measurement technology. 
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■ \ - ■ •- ' 

^ \^ Attempts t,o make f he goals of education clear and measurable or to 

narrow the range of wh^t schools flire expected to accomplish have en- 
■f . - ■ / - ^ 

^countered several difficulties; Making the goals specific often results 

■' ' 

in heated contests among various community or school factions. Limiting 
the range of goals collides yith the expectations various individuals or 
grqups^hold for schools. For example, focusing on th^in>i;^uctional or 
socializat^^bn functions of education ^oftpn runs afoul otV^other^ more 
latent functions such as custody control, evaluation, certif ici 1:ion, and 
selection (Spac --"^^ * 
^^ The special qtialities of educational goals have important ii^plica- 
tions for formative eValuition in schools. In fact, it is nearly^: 
possible simply, to begin fc^m^ive evaluation with a statement oJt-^duKa- 
tiOnal goals and then tcf measure the extent ►to which they have been 
realized. Rather, the formative evaluation process itself should pro-* 
Vide the basis for reaching a consensus on goals or on tne problems 
that confront the 'system. I^*oll6wing this initial stage, the criteria 
for measuring goals and other indicators ,pf school performance may then 
become the focus of subsequent evaluation. 

Although the special qualities of educational environments and 

..goals have the most iigportant implications for formative valuation in 
schools, th6 other org^i^zational subsystems ajso have qualj^ties that 
affect evaluation activities. The unique character of educa\iional 
technology^ group processes and norms the formal structure of 
schools contribute additional problems that make formative evaluation 
difficult. V ' > 

An organization's technology is a series of integrated activities, 
procedures, or processes conducted to accomplish intended goals. Under- 
lying a. technology is a set of beliefs about the linkage between the " 
activities or procedures and the intended outcomes. In^ schools, the 
main technology (instruction) is fragmeaa^d, and the link between 
instr^uctional activities and learning outcomes is relatively weak" . 
CUrri'cula, instructional packages, and t^achijni'g strategies are ex- 
amples of educational technologie^s. But for examples the knowl- 

* edge* or beliefs to sup'port a specific cause-effect linkage between the 
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technplogy and the outcome do not ;exist (Oreeben. 1970). The linkage 
betwee^n teaching strategies' and learning outcpmes is not understood. 
' For this reason, evaluation that focuses solely on educational outcomes 
does hot provide a dii^ction for improvement. Reporting achievement 
%cores tb teachers or schools does not, by itself, provide a direction 
for change. Student performance on such tests is only an indicator and 
^ unless accompanied by informatlan on other aspects of instruction or 
^ organization of the school is relatiyely useless for formative 
purposes. 

'—-As in all organizations, individual and small group activities are 
an essential subsystem. Around these activities informa-l relationships 
and tiorms arise which influence the formal structoire of the organization 
and th^vay work is performed. Such inform al relationships and norms 
are often quite powerful in schools, parti cularly at the teacher level, 
^ .^d are buttressed by teachers asso ciations, which emphasize the teacher 
r;''^ as. an autonomous professional . These two factors combine to produce the 
V 'bureaucratic-professional conflict often found in organizations. In the 
absence of well-developed formal controls or sanctions, informal norms 
and processes are particularly potent. They are often powerful barriers 
to change and are resistant to evaluation efforts. They must, however, 
bi^ part of formative evaluation if it is to accomplish its main goal- 
improvement of the system. 

Finally, the formal structural features of schools are unique. 
Witkn school districts, for example, the various organizational levels- 
district, school, and classroom—operate independently of one another 
with little formal coordination or control, particularly in the area of. 
instruction (Deal, Meyer, and Scott, 1974; Meyer, Scott,. Intili, and 
Main, 1974). Within levels also, participants operate independently. 
There is little formal work-related interaction within schools either 
among teachers or between teachers and specialists (Cohen and Bredo, 
1974) . There is little interaction between principals or teachers of 
different schools within the same district (Meyer,, Scott, Intili, find 
Main, 1974) . Evaluation of teaching or instruction is virtually non- 
existent (Dornbusch and Scott, 1975). Formal evaluation of teachers 
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or pflnclfials Is not conducted frequentljr. ^Neither, in most schools, do 
teachers evaluate each other^s cla^sreom performance. Structurally , 
schools are loosely coupled collectives rather than Well-knit formal 
" or^nizations . This Arrangement minimizes the costs of coordination, 
'such as time and conflict (Deal, 1975). Formative evaluation in schools 
must recognize these existing structural patterns and the National basis 

for their maintenance. At the same time, these patterns of social or- 

• ' 'f 

ganization mu?t: be a pripoe focus of formative evaluation. 

To date^ formative evaluation in education has not been directed at 
.all the important organizational subsystems in schools. Nor has it 
recognized the sfpecial characteristics of these various subsystems and 
their dynamic interrelationships. Few evaluators have focused on the 
organizational stiructure of schools;; few have focused on environmental 
configurations and pressures; few have emphasized instructional techniques 
or proce'dures or the linkage between existing teaching strategies and 
educational outcomes; few -have focused significant attention on individual 
and small group processes and norms. While most educational evaluations 
have begun with educational goals and measured the extent that they have 
been realized, insufficient attention has been paid to the peculiar 
nature of educational goals, which makes sudh an em^asis problematic. 

In sum, attempts, at formative evaluation 'in^jeducation have not fo- 
cused on the five important organizational subsystems and their inter- 
relationships. Instead,^ the emphasis has been on instructional evaluation 
OP the measurement of educational outcomes even tliough this approach, 
given the pecul;Lar nature of school organizations, is usually less than 
effective. This^ emphasis has distorted the reality of highly complex 
social organizations. It has not produced the feedback that is needed 
if schools are to develop coherent, effective educational programs and 
design social organizations that provide suitable support. The narrow 
i c'is has contributed to pa^ternSkf school organization that are in- 
suiricient to cope with both complex educational environments and so- 
i AkSticated instruction (Deal, 1975). 
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As formative evaluation is applied in schools it must take into 
account the unique nature of the educational enterp^se. In addition, 
this application must recognize general problems in the field of forma-r 
tive evaluation. It must therefore: (1) be related to the decision- 
making process, (2) involve all participants irv^he evaluation process, , 

(3) make explicit a dynamic, systems-oriented^^eoretical framework, 

(4) consider the social and political reali4:ies df the school's "environ- 
ment, and (5) emphasize the special characteristics of school organi- 
zations and their important relationships,. Formative evaluation in 
schools cannot merely address instruction? must consider the organi- 
zation if schools are to identify and taake necessary changes and provide* 
for their supp\)rt. / 

i 

/ 

Survey Feedback as a Fonnative Evaluation Technique 

■ ■ ' 

The Environment fqr Teaching Program at SCRPT has developed an 
alternative strategy designed to overcome the limitations of formative 
evaluation in school organizations — a survey-feedback approach (Deal, 
Duckworth, and Robbins, 1975)V\ 

We begin the ^proceSs with a survey which gathers systematic and 
comparable information from the ^participant — administrators, teachers, 
students, parents, and members of the community. This survey seeks 
information in such areas as educational views and preferences; the 
present instructional program; the relationship between the school and 
the community; the work relationships among teachers and between teachers 
and the administration; student attitudes, aspiratiQhs and preferences; 
the current decision-making and problem-solving processes; and the. 
satisfaction of all participants, both overall and with respect 'fefc spe- 
cific Issues. Since a wide variety of views about many issues is ob- 
tained (at various levels within the system, and from various roles and 
from people outside the system), the survey produces a solid body of 
information both about the current state of affairs and about desirable 
directions for change. The survey information, together with information 
on student achievement, demographic data, and other existing information. 
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provldes, a base for evaluating the school that ^includes the perceptions 
of all participants as veil as facts.. 

The second step in oifr strategy is to introduce a system^ view of . , 
the dynamic relationships between a ^dhaol and its community; between 
the instructional program and teachers, studepts, and the administration; 
and among the various roles that are required ^to make an educational 
program or 9chool run. The distinctive characteristic of this perspective ' 
is its emphasis on organizational attributes, rather |han individual 
characteristics, as the source o|i|problems and of resistance to proposed 
changes* ^ - „ 

^ The organizational view of the process of change is introduced to 
participants of a school or school district through a planned workshop. 
Tm workshop uses lecture, discussion, participa'tive exercises, aiTd 
Materials to instruct school personnel in the organizational approach 
^o problem solving and change. It focuses on. general concepts and theory. 
It introduces participants to the five organizational subsystems and 
their relationships to one another. It also introduces .participants to 
specific skills such as brainstorming, leading ^^M^^sions, solving* 
problems, interpreting survey data, making infereiTCBS from data, and M 
developing criteria for selecting change strategies. These perspectives, 
skills, and techniques enable the participants' to explore and eventually 
use the information provided by the survey to evalCiate the existing 
situation, to pinpoint trouble spots, to develop directions for change, 
and to anticipate the support required for changes, as well as to pre- v 
diet any likely resistance. As an illustration, we might find that 
community educational preferences were not incorporated in the existing 
instructional program. This analysis might lead to suggestions for 
instructional changes. But the instructional changes would, if neces- 
sary, be accompanied by changes, in the structure of the school or in 
the individual or small group skills needed to support the new ins true- 
tional approach. At the same time, the analysis might predict that 
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certain s,egments of the co^nraunlty oi|||pt/gra)up of teachers within the 
school would oppose the changes. Strategies would then be developed to 
ensure ^ihat these views were heard and Incbrporated In the' changes 
ultimately made. 

The third step In'' our survey-feedback strategy Is to emphasize 
participation In the ^)rocess of evaluating the school and determining 
directions for Improvement. All/rt:^se who will be rpotentlally involved^ 
in or affected by changes are Inwlved in th©f proces^ Thus, the views 
of all will be heard, and the legitimacy Of the evaluation as well as 
the level of commitment to the proposed change will be increased. 

' ^ The dilemma'of' full participation is to involve everyone with rele- ; 
vant Interests withoUt' creating a chaotic, unproductive squabble among 

\ r 

them. We circumvent this problem by having two kinds of groups which 
will, help conduct the survey, feed back the results^^and discuss action 
strategies. A policy group is formed from representatives of all the 
constituencies involved; e.g. district administratfSrs,^,:^ the principal, 
teachers, parents, and students. Peer groups ate also formed and are 
composed of individuals with comnion roles or interests in the school, 
e.g. the members of individual departments or grad^levels. Groups of 
parents or community representatives can also be formed into 
group, as can students^ The peer groups'^meet to discuss the survey 
findings most relevant to themselves, and to feed back the results of 
their discussions to the policy group. The policy group thus has some 
idea of the desires of different interest groups in the school and the 
community as well as having access to the overall results of the study. 
This group may then consider alternative policies which could be Instl- 
tuted, and suggest strategies to the pee* groups for their review. In 
this way a dialogue is begun between the peer and policy groups, between ^ 
the special and the general Interests, flie dialogue will help gather 
Inputs for problem solving as well as generate commitment to the ongoing 
process. In this way it will Institutionalize problem solving in the ^ 
system. An<f the evaluation activi^ties will be linked to the ^xisting 
decision-making apparatus. 
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The combination of objectiye information, an prgani'zational per- 
spective, and the participation propess make survey feedback a unique 
approach to formatlye evaluation. Activities similar to those we piror 
pose are now used by schools or school districts: needs assessments, 
organizational^ climate surveys, organizat'fonal development prpgramsr, 
instrucb^^^^^sessments, and community involvement programs/ Suij^ey 
feedback^T^^dnlque because it provides a planned sequence for rela^ng 
these -Otherwise diverse activities In a systematic way. * ^ , ^ 

ftow It Works . • 

Three groups are involved in the survey-feedback process. We haVe 
mentioned the peer groups, which ar^. natural work grqtips in the school, 
and the policy gr'aup, which is composed of representatives from all 
relevant constituencies. The third group is the advisory team.' This 
group is analogous to the outside evaluator or consultant, and in our 
case includes representatives from the staff of the Environment - for 
Teaching Prqgram at SCRDT. , ^ 

The policy group, the peer>^ groups , and the advisory team form a 
temporary problem-solving structure. Each group has specific responsi- 
bilities as the survey information is reported and used.' 

The role of the policy group " , s . 

1. Reviews questionnaires and determines field procedures. 
2,. Provides legitimacy for the study among various constituencies 

3. Participates in problem-solving workshops. 

4. J,. Uses survey results and input from peer groups to define 
school-wide problems. - 

5. Develops school-wide change strategies attd oversees their 
implementation. 

6. Takes an active role in evaluating the results of change 
strategies. 
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The role of the peer groups ' . . 

1, Discuss survey information that pertains specifically to 
the work group, ♦ 

2, Participate in problem-solving workshops, 

t» ^ 

3, Define the problems of the work gr<jup and develop tentative 
strategies for solving • them, ^ ' ^ - '■ \ 

4, Discuses the schaol-wide problems s^utions identified hf 
the policy group. 

5, React to proposed school-wide 'changBS, 

6, Implement both specific peer group and school-\H.de solutions. 

The role of the advisory t^^m ' 

1. Provides criteria for selecting the policy group. ^ 

2. Provides *survey instruments and \torks closely with the poiSfty 
group in developing field .procedures . 

3. \ Supervises, the collection of information. 

A. Analyzes information and highlights possible problem areas 

5, Conducts problem-solving workshops to provide a common 
framework for discussing information, defining problems, 
and^ proposing solutions. Trains di^tvssion leaders for each 
pe«c group , " . y 

6, Advises on process and suggests alternative formulations as 
prolilems are defined and solutions- are proposed, 

7, Assists in determining the effectiveness of change g^trategies 

The survey-feedback process .in a school unfolds in the following 
sequence: . ' 

Step 1, Orientation . * ' 

Advisory team introduces survey feedback to faculty, administra- 
tion, and other groups; discusses the formation and composition^ 
of the policy group (the desired roles); And suggests procedures 
, for selection, ^ . 

Step 2, First survey design meeting 

Policy group meets with advisory team to (a) identify relevant 
target areas, (b) Identify groups \ to be, surveyed, (c) discuss and 
assist with informA information gathering* 
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Step 3« Secorid survey design meetii>g » . 

Design procedures for gathering infSrinatibn.Y Adviso^ team meets / 
with policy grdup to (a)' review instruments and (b5) establish 
procedures for collecting data. ^ \^ ^ . 

Step 4. Data collection ^ ^ 

School staf^ students, distrit:t personnel, and patents complete 
survey. ^ ♦ • , 

Step 5. Data analysis and preliminary diagnosis 

Advisory team analy:&es data, defines problem areas, assesses 
"match" among preferences, instructionalf program, school organi-* 
zation, and environment. • f 

Step 6> Probl^mfsolvinfef'vorkshop 

AdvisQTcy team meets with policy group^^ 
(a) .introduce organizaytonal approacB^^ 
(by train discussipA leflHers for feed| 

Step 7. Feedback to peer -groups^ / ^ .^^^^^v 

Advisory team meets with peer grhup lead€flra -to feed back relevant 
results concerning their groups. ' ^ ^ 
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Step 8, Feedback to policy groups 

Each pdei; group leader repj^fts his/her -group's results from survey 
and strategv suggestions. Advisory team presents overall results 
and firidingb. Entire group discusses Organizational strategies, 
proposals for changes, etc. , 

Step 9. Feedback to peer groups 

Peer group leaders again feed back .overall findirigs and suggestions 
from policy group meeting, especially as It concerns implications 
for theif particular group, ar^d discpss organizational strategies. 



Step»10. Strate53cy Session 



Policy group and advisory team meet to discuss and confirm actual 
plan to implement strategies. 

Step 11. Evaluation of results \ 

After a specified time^ parts of the survey are readministered. 



20 



-18- 



Intended Effects * 

J In essence, the survey-feedback process Involves a comprehensive 
(irganlzatlonal analysis of a school. It gathers information from the 
fjWe important organjf.zational subsystems and uses the information to 
evaluate the existing sitliation and tjo provide a basis for making im- 
provements. From the very beginning, the school's participants are > 
heavily involved. The main goal of the process is to pirovide the 
participants with information, problem-solving skills, and a sequence 
of activities that will enable them te-^improve their existing situation. 
^ At. the same time the process emphasizes fihe importance of anticipating 
resistance to changes and proyiding the support that is necessary to 
maintain changes over time, ''tike all formative evaluation, survey feed- 
' back is a diagnostic technique. "Unlike other formative evaluation, 
however, it avoids some recurrent problems^ Let us look mare closely 
at how survey feedback addresses jthese jx^eviously identified problems. 

The Potential of Survey KeedbacV as a rermative 

Evaluation Strategy , ^ 

^ Thus far We have describaij:he problems and needs discovered from 
ah dxamination\>f the lit^^ture on formative evaluation; the unique 
problems' entailed in evaluating schools; and a specific fbrmative evalua- 
tion- technique, survey feedback. Keeping in mind the special problems 
of school organizations, we shall now use the four needs of formative 
evM.uation as criteria for assessing the value of survey feedback. ^ 

\ Need for continuous assessment and feedback to decision makers . 
Thks is the basic premise upon which the survey-feedback approach is^ 
based. Th^^^feedback sessions lead directly to strategy and implementa- 
tion sessions. Strategies are then assessed in terms of their , ability 
to sfolve the identified problems. The cycle is thus .complete. Each 
aspect of the process is given equal importance. Relevance to the 
decision-making process is not left to chance or good intentions, it is 
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built In. I'dealiy, formative evaluation is a continuing prodess'. In 
survejr feeibVck, the last step of the first cycle automatically becomes 
the' first step of the next... ad infinitum. ,' / 

/ This concern for continuity is especially crufeial.ln such ongoing, 
programs as schooling. In the survey-feedback ap/roach the institu- 
tionalization of formative evaluation as a continuous data gathering and 
feedback system is not simply an ideal—it is expected. fj 

. Nfeed to involve relevant groif^)s . Taylor knd Maguire h^e^Wnted 
to flie groups (and we would add a sixth, administrators y having im- 
portant opinions about education: spokesmen for society at large, subject 
matter experts, teachers , .parents , and the students themselves. Members , 
of- these groups .are educational judges who." should be heard. Superficial 
polls, letters to the editors, and other incidental activities are in- 
J sufficient' to obtain these judgments. An evaluation of a school program, 
should systematically take into account its merits and faults as perceived 
by well-identified groups. ' Thus , judgment data and description data are 
both essential to the evaluation of Educational programs (Stake, 1967). 

Judgments from all these groups are integral to the survey-feedback 
approach. *The usual reliance on biased information, hearsay, and rumor 
^re replaced by systematic information solicited through a participative 
process. Not only should the quality of the evaluation product be en- 
hanced as a consequence, but the commitment to any resultant change 
strategies should be increased. 

The schoolVs unique organizational characteris^tics—the importance 
of its loosely Lupled formal structure (Meyer, 1975) and the' dependence 
on the environment—can all be taken into account in the survey-feedback 
process through the mechanisjtis provided for input by the different kinds 
of groups and by^ncluding these various interests integrally in the 

process its^f. . ; 

Another problem in formative evaluation addressed by the survey- 
feedback approach is the advantage of being able to conduct the evalu- 
ation internally. After a school has completed the survey-feedback 
process initially, it can- be continued as an ongoing evaluation technique 
by "school personnel, thereby eliminating further, dependence on external 
resources . 



22- 



-20- 



Need to make the theoretical f^mewoi^k >explj.cit » ^ Not only does a 
dynamic organizational theory underlie the survey-feedback process; it 
is explicitly transmitted both in the orientation session and ^ the 
organizational problem-solving workshop • Participants therefore do not 
have to guess at the assumptions and purp^oses of the evaluation; the 
underlying framework is systematically shared with them. Like Lawrence . 
and Lorsch's (1969) contingency model of organizations » our approach 
overtly seeks a '^t" amoyg the instructional program, the structure, 
and the environment of tl^ school organization. This congruence among 
the three levels is seen as instrumetital in leading -to the effectiveness 
of* the organization.' This framework is also consistent with Etzioni's ^ 
djmamic systems model. His approach ^nipiiasizes the effectiveness of such 
organizational functions as "recruiting' tesburces, maintaining the 
structure, and achieving integration into the environment" (in Weiss, 
1970) rather than overall goals. 

The framework underlying the survey-feedback approach considers 
the important organizational subsystems and their interrelations.. 
Etzioni's functions of an organization overlap with Lawrence and Lorsch's 
and Udy's subsystems. A dynamic systems approach provides the icey to 
iluating the school as a total organization. It also minimizes the 
importance of setting the goals at the outset. This is especially 
crucial in a context where the goals are ambiguous, fined, and 

Need tK> "^ons^jofe^ the social and political context . The survey- 




feedback approach addresses this concern on two levels. First, the 
process provides structural mechanisms for soliciting input from the 
environment and for generating output to it. Second, during strategy 
development and implementation, supportive and constraining forces of 
the social and political context are considered. Explorations can be 
made as to what is politically and socially feasible and cost-effective* 
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As. we have pointed out i^re^iousi^^, education not only occurs within 
such a social and political context but'Hs dependent upon It for Its 
v4;jry -existence. . There Is no question whether to consider this context; 
in evaluation; its inclusion is essential to survival/ The survey-feed- 
back approach recogjiizes /this Imperative and systematically includes the 
envlroiment in the fbrmacjive evaluation process. 

Problems and Prospectg 

Lest the survey-feedback process be seized and embraced as an' 
educational curerall, we hasten to emphasize that it^is not/ As with- ^ ; 
any evaluation strategy joper^a ting in the real world of schools, there 
are potential problems. These exist in the process Itself, its under- . 
lying assumptions, its appiic^bllity to a particular setting, .and. costs. 

Within the survey-feedback process, there are three possible 
problem areas: securing real representativeness In the policy group, 
fostering an adequate dialogue between the poli'cy and peer groups, and 
cultivating the organizational perspectives and special ^skills under- 
lying the process. The last of these is most problematic. Given the 
individualistic or psychological view predominant among school admin-, 
istrators and teachers (Deal, 1974);, it may 'prove difficult to encourage 
participants to viJI their world "organizationally." There are also 
specific skills in translating survey results into change strategies or 
improvement plans, but ap^rt from the obvious discussion, problem 
solving, and group process skills, the linkage between assessment and 
solution may not be readily apparent to participants. The problem- 
solving process requires a familiarity with cause-effect relationships 
and knowledge of how to use systematic information to define problems 
and develop solutions. Bridging the gap between an academic approach 
and a real situation is. difficult under any conditions. 

Even assuming that the process unfolds according tp the ideal, other 
problems ma]^ interfere. The first is in the area of educational goals. 
Most existing evaluation approaches begin by soliciting the organiza- 
tion's goals as evaluation criteria. The survey-feedback technique, on . 
the other hand, assumes twQ; levels of goals. The first level is process 
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goals and emphasizes developing a consensus on pj|pblen^ and change 
strategies which then become goals or evaluation criteria, H^ince, at ^ 
the first level, the "goal" of the technique is goal consensus. This 
emphasljs, because ^oi^xtts ambiguity, may. be difficult for those in educa- 
tion wno , are .accustomed to beginning an e^^^luation with a statement of 

■ ' ■ .0 
goals, then collecting J.nformation to measure their attainment. The 

second level of goals broadens 'the range of typical educational evalua- 
tions and specifies that the ultimate goal of schools is overall organi- 
z^tional effectiveness, -inciuding participant satisfaction and community 
su^ort as well as the realization of fttudent leSraing outcomes. This 

id' level of goals is related to the organizational perspective under- 
lying the survey-feedback process. This, once- again, may be .unfamiliar^ 
to participants. The organizational perspective assumes that brganiza- 
"^tional effectiveness^ is attained by a "fit" or "equilibriiim" among the 
various organizational subsystems^. This theory is essentially a 
contingency view of organizations and assumes that effective organiza- 
tlons-^are those that have designed structural features to fit their 
environment, program, and informal processes. ^ • 

A second problem coiild exist because this underlying theory^may be 
inadequate or may not lend itself to a particular school situation. 
Pjirther, even if the theory adequately captures, or orders reality in a 
^iven context, a valid measurement ^f the varicfus concepts in the survey 
may be outside the grasp of our instruments. 

A'ltftlrd potential problem may arise in the "implementation stage. 
As with most evaluation systems, the survey-feedback process assumes a 
logical, rational orientation on the part of the partlclpantg. It 
assumes that all alternatives, preferences, and consequences are fully 
kriown and that participants will choose rationally among them. What may 
be closer to actuality is that all alternatives ^ preferences, and con-"^ 
sequences are not fully known. Even if knowp^ the- participants may not 
choose, rationally. And finally, a logically chosen strategy may be 
undenained by. unanticipated political and social realities of the situa- 
tion. Despite these constraints, however, survey feedback can be useful 
as a way of increasing the "known" part of the equation and anticipating 
t^e social and political ramifications. 

■ 3 • . ■ 
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A fourth problem Is that the survey- feedback process Is expc 
(ierr, 1974)^. Retaining' the outside advisory team to administer the 
survey, conduct workshops^ and participate as' a full partnjpr in the 
process costs npre than many schools or school districts are presently 
willing to invest in formative evaluation.' Even if a schooL district^'" 
•is willing to -commit the financial resqurces, obtaining qualified 
persontfel with backgrounds in survey research, organization theory, and 
the skills needed to facilitate the problem-solving process may prove 

difficult. t . . ' " 

'The time required of teachers, administrators, parents, and com- 
mvftiity members ife also substantial. Administering the survey, completing 
the survey, participating in workshops., solving problems in peer groups, 
and serving on the policy group require that participants spend more time 
than usual in evaluation and in planning activities. We estimate that 
the survey-feedback process will require at least eight wpeks to complete. 
If this expenditure of time is viewed as taking too much time a^ay from 
• the day-to-day operation of the program, schools wil)! be reluct|^t ta 
participate. , ' ^ 

Moreover , the process is bound to produce. more-c;onf lie t than would 
be the case in "business as usual," where many importaW conflicts are 
smoothed over or avoided,. survey-feedback' process ,1 by making latent 

conflicts explicit, may initially produce power struggles, infighting, 
and interpersonal stress. When this is added to the conflicts that will 
surface as new patterns of social organization are created, the process 
may appear too expensive. Buffering, as a means of responding to complex 
environments or instructional complexities, is cheap in terms of time and 
conflict • (Deal, 1975), particularly "when many of the costs of buffering, 
such as reduced organizational effectiveness, are as difficult to assess 
as they are in schools. Only if th^^ costs of survey feedback are com- 
petitive with the costs of just enduring will schools be likely to 
undertake it. Even though survey feedback may, in the long run, be more 
effective than most formative evaluatioi^*^ its short-term costs appear 
high. Although schools presently spend money and time for the separate 
activities included in the survey-feedback process despite the costs, 
these various^tivities are rarely related in a systematic, global process. 
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: ATtho'ugh It^ tl^e survey-feedback process addresses 

•some very fundamental concerns repeatedly found in the area of formative 
evaluation'.. -It^^dkes into account the special characteristics of schools. 
It directly bridges the gap between diagnosis' and action. It ties to- 
gether the presently discrete areas pf organizational development, form- 
ative evaluation , and educational research . Perhaps , owing to the survey- 
feedback approach and others like it, it will someday be said that 
formative evaluation is effective in fo%tpring Educational improvement, 
and is not like water running off a duqgs back. \ 
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